The Estimation of Distributions and the Minimum Relative Entropy Principle
نویسندگان
چکیده
منابع مشابه
The Estimation of Distributions and the Minimum Relative Entropy Principle
Estimation of Distribution Algorithms (EDA) have been proposed as an extension of genetic algorithms. In this paper we explain the relationship of EDA to algorithms developed in statistics, artificial intelligence, and statistical physics. The major design issues are discussed within a general interdisciplinary framework. It is shown that maximum entropy approximations play a crucial role. All ...
متن کاملThe Factorized Distribution Algorithm and the Minimum Relative Entropy Principle
Estimation of Distribution Algorithms (EDA) have been proposed as an extension of genetic algorithms. In this paper the major design issues of EDA’s are discussed using an interdisciplinary framework, the minimum relative entropy (MinRel) approximation. We assume that the function to be optimized is additively decomposed (ADF). The interaction graph GADF of the ADF is used to create exact or ap...
متن کاملA minimum relative entropy principle for AGI
In this paper the principle of minimum relative entropy (PMRE) is proposed as a fundamental principle and idea that can be used in the field of AGI. It is shown to have a very strong mathematical foundation, that it is even more fundamental then Bayes rule or MaxEnt alone and that it can be related to neuroscience. Hierarchical structures, hierarchies in timescales and learning and generating s...
متن کاملThe Minimum Entropy Production Principle
It seems intuitively reasonable that Gibbs' variational principle de termining the conditions of heterogeneous equilibrium can be gener alized to nonequilibrium conditions. That is, a nonequilibriurn steady state should be the one that makes some kind of generalized-entropy production stationary; and even in the presence of irreversible fluxes, the condition for migrational equilibrium should...
متن کاملA Minimum Relative Entropy Principle for Learning and Acting
This paper proposes a method to construct an adaptive agent that is universal with respect to a given class of experts, where each expert is designed specifically for a particular environment. This adaptive control problem is formalized as the problem of minimizing the relative entropy of the adaptive agent from the expert that is most suitable for the unknown environment. If the agent is a pas...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: Evolutionary Computation
سال: 2005
ISSN: 1063-6560,1530-9304
DOI: 10.1162/1063656053583469